Repository navigation
perf(server): a folder's repo root is remembered, not asked of git on every lookup - #1037
Merged
Merged
Conversation
… every lookup RepositoryIdentityResolver ran `git rev-parse --show-toplevel` on every resolve, because git's answer was the identity cache's key. The pull request sweep alone made 32 of those per pass over 11 folders, every 20 to 120 seconds, about a third of the child processes the server starts. The resolver now remembers each folder's root and confirms it on disk: the folder still resolves to the same real path, no folder between it and the root holds a `.git` or a `HEAD`, and the root's `.git` exists. When a check fails it asks git again at once, and it re-asks every 10 minutes regardless. An answer the disk can't confirm is never reused. Concurrent resolves of one folder share one git run. A sweep now makes 11 lookups on its first pass and none after.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
RepositoryIdentityResolverrangit rev-parse --show-toplevelon everyresolve(cwd), because git's answer was the key of the identity cache. So the cache never saved that process. On a real install the pull request sweep made 32 of these per pass over 11 project folders, every 20 to 120 seconds. Together with the other callers it was about 18 of the 52 child processes the server started each minute.Fix
The resolver remembers each folder's repository root and confirms it with file checks instead of starting git:
.gitor aHEAD(a bare repository),.gitstill exists.When a check fails, the next resolve asks git again at once. Remembered roots are also re-asked every 10 minutes. An answer the disk can't confirm (git timed out, a symlink git resolved differently, a
.gitgit skipped) is never reused, so those folders behave as before. Concurrent resolves of one folder share one git run. The fix is in the resolver, so every caller gets it.How fast a change is noticed
git initin a folder with no repocore.worktreebeing setThe last row is the trade this change makes.
Tests. New tests count git runs through the process runner: repeated and concurrent resolves, the 10-minute re-ask, a nested repo, a nested bare repo (fails without the
HEADcheck),git init, and a folder re-created as another repo. The resolver's 12 tests pass, along with the 462 tests in the files that build the real resolver, and fmt, lint and typecheck.Review. An independent review of the diff found three things. Fixed: a bare repository created inside another repo kept the parent's identity for 10 minutes. Fixed in code, not run on Windows: git prints
//server/share/repofor a network share while the filesystem says\\server\share\repo, so the two never matched and the lookup was never remembered; the separators are converted before comparing. Left as is: the bounded cache can evict an entry whose lookup is still running, so two lookups for one folder can overlap if 512 other folders are resolved in the meantime. Before this change no lookups were shared at all.Not checked. A run on Windows, and a live server.